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ABSTRACT 

This study investigates relationships between 
differences in students' achievement and differences obtained by a 
technique held to yield a reasonably valid representation of 
students' memory structures of a particular subject matter. Also 
investigated was the nature of the flow of information between 
curriculum materials, teachers, and students. This was achieved by a 
comparison of the digraphs of content structure and the graphs 
constructed by the teachers and the students. The subjects were five 
mathematics teachers (grades 7-10) and their six classes (student n = 
109) ♦ The content used was a curriculum package on operational 
systems which involved one lecture and a seven-page booklet for 
students to study. Students and teachers on the average reflected the 
content structure of the subject in their constructed graphs. There 
were differences in achievement corresponding to the correctness of 
srudents" graphs of the content structure, but these differences also 
reflected variations among the teachers' graphs, (JP) 
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In recent years, mathematical curricula have tended to move 
away from emphasizing rote learning and computational skills and 
toward greater emphasis on teaching subject matter structure (Bruner, 
1960; Klepfer, 1971; Schwab, 1962; Wilson, 1971). This shift in em- 
phasis has been accompanied by a variety of justifications, including 
claims that instruction" oriented toward learning the structure of 
the curricular material leads to a fuller understanding of the sub- 
ject matter, which in turn results in enhanced achievement, increased 
problem-solving ability, and transfer to new situations, as well as 
enhanced retention and increased student enthusiasm for the subject 
matter. 

Although such expected outcomes of the structure-teaching ap- 
proach are admirable and of considerable educational value, there is 
not a great deal of empirical evidence to support the claim that these 
outcomes do actually occur (but see Grceno, 1972; McKenzie, 1972; 
Mayer and Greeno, 1972; Shavelson, 1972). This study is an attempt 
to investigate relations between differences in student achievement 
and differences obtained by a technique held to yield a reasonably 
Oj vrlid representation of students' memory structures of a particular 

subject matter. 



The differences between an achievemGut test and a representation 
of cognitive structure should be made explicit in regard to the kinds 
of data obtained. An achievement test usually provides evidence for 
such student variables as precision of discrimination between concepts, 
accuracy of conceptual definitions, and appropriateness of conceptual 
application. However, a valid representation of a student's cognitive 
structure should indicate how a student associates key curriculum 
concepts, which clusters of concepts are considered relatively impor- 
tant, and the degree of inclusiveness of conceptual clusters. Such 
a representation could then provide the reference for data bearing 
on assumed correlates or consequences of attaining an appropriate 
structure. As an example of such a correlate consider the question: 
Do students with memory structures corresponding closely to the sub- 
ject matter structure achieve at a higher level than students evi- 
dencing a lower degree of such correspondence? A valid representation 
of cognitive structure could also be employed, in conjunction with 
representations of content structure, to analyze results of the flow 
of information between curriculum materials , teachers, and students. 

METHOD 

Subjects : The subjects were five mathematics teachers and their 
six classes (student n^ = 109) , The teachers were completing a fifth- 
year teacher training program leading to state certification as mathe- 
matics teachers. The students ranged from 7th Co 10th grade level, 
and represented a range of socio-economic status, as well as a range 
of mathematical proficiency. 
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Instructional Material ; The primary content of this study was a 
curriculum package on operational systems. An operational system is 
defined as a set together with a binary operatio). on the set. An 
operational system may possess any, or none, of the following funda- 
mental properties: associativity, corranutativity , identity element, 
and roundness. This content was selected because operational systems 
is a mathematical structure including concepts that can be hierar- 
chically arranged, and because concepts embedded in the' operational 
systems curriculum are fundamental to many other mathematical struc- 
tures and are, therefore, mathematically significant. 

Employing material developed by the Secondary School Mathematics 
Curriculum Improvement Study (1967) and the Comprehensive School 
Mathematics Project (1970), two mathematics curriculum specialists 
(Branca and Pelavin) developed the actual operational systems curri- 
culum package, which consisted of an hour lecture and a seven-page 
text. The text gave several examples of operational systems and 
summarized the material discussed in the lecture. From this package, 
the following twelve concepts were identified as crucial: associa- 
tivity , binary operation , commutativity , element , fin i te/ infinite , 
fundamental properties , identity element , inverse , operational system , 
ordered pair , roundness , and set . 

Figure 1 presents a hierarchical graph representing the relational 
structure, or organization, of these key concepts that was obtained 
from the curriculum developers. They considered this representr^ion 
to be correct and meaningful. 
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Instrumentation ; The curricuJ.uin developers', teachers' and students' 
cognitive structures were examined by first administering a linear- 
graph-building task (Fillenbaum and Rapoport, 1971). In this task, 
subjects v/ere provided a list of the key concepts. They then selected 
wnat they felt were the tv/o most similar concepts and connected them 
with a line (which they labeled "1"), thereby forming a cluster. 
The remaining concepts were similarly associated, either among them- 
selves or with clusters already established. Then a proximity matrix 
was formed, using the sums of the numbers on the lines connecting ecxch 
concept with every other concept as an indsx of the intra-conceptual 
distance. This proximity matrix served as i..put to Johnson's (1967) 
hierarchical clustering (HICLUS) program, which yielded a represen- 
tation of the hierarchical structure underlying the linear graph* 

Two parallel forms of an achievement test were constructed for 
this research project. One form was administered as a measure of 
pre-treatment achievement, and the alternate form was used as a mea- 
sure of post-treatment achievement. Each form contained items covering 
the 12 key concepts, with the items spread across three levels: (a) 
knowledge of definitions of key concepts, (b) familiar problems, and 
(c) unfamiliar problems. 

The content structure of the curriculum was analyzed by a digraph 
method (Shavelson, 1972, 1974), and yielded the HICLUS representation 
presented in Figure 2* 

Procedures : The curriculum developers held an instructional session 
for the teachers, at which time the lecture developed on the opera- 
tional systems curriculum was given. The seven-page text was also 
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given to the teachers at this time. After the session, one form of the 
achievement test was administered to the teachers. Subsequently, 
the teachers prepared lesson plans based on tlie material covered in 
the session with the curriculum developers and in the seven-page text. 
Next, the teachers administered the graph-building tajk and one form 
of the achievement test to their students, spent about three class 
periods teaching operational systems, and then re-administered the 
graph test along with the alternate form of the achievement tes^^ . 
About a week later, the graph-building task was administered to the 
teachers . 

RESULTS 

The achievement scores obtained from the teachers were essentially 
at ceiling, supporting the cor elusion that they understood the opera- 
tional systems material according to this manner of measureing com- 
prehension . 

Figure 3 presents the HICLUS hierarchical structure resulting 
from the combined dissimilarity matrices obtained from each teacher's 
linear graph. This hierarchical graph is almost identical to those 
in Figures 1 and 2, indicating that the teachers' cognitive structures 
were, on the average, good and correct. The flow of information 
from the curriculum materials to the teachers underwent no significant 
distortion observable in this grouped data. 

Figure 4 presents the lilCLUS structure obtained from the com- 
bined linear graphs of all students prior to instruction, and Figure 
5 presents the post-instructional structure for all students. Al- 
though very fine-grained comparisons between graphs are probably 
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not justifiable, it seems clear that there are some reasonably large 
differences between the two graphs that give evidence of change in 
the direction of greater congruence v/ith the *'ideal" structure rep- 
resented in figures 1,2, and 3, For example, at the pre-instructional 
stage, the students do not connect the concept of identity element 
with the roundness and inverse concepts. This problem is not mani- 
fest In the post-instructional graph. Also, at the pre-instructional 
stage, roundness and inverse enter very late into the strucCure. 
Such a result v/ould occur if these concepts were not connected v/ith 
the other concepts until the final steps of the graph-building task. 
Students not understanding these concepts would be expected to behave 
in this manner. At the post-instructional stage, however, these 
concepts form suLclusters at a lower level, i.e., they are collected 
into sub-clusters notably earlier than the point at which the sub- 
clusters are interconnected. As v;ell, the post-instructional graph 
indicates that the students, in general, have learned to connect 
identity elenient into a sub-cluster with roundness and inverse . 
Also, tVie post-instructional clusters seem to be formed more directly 
and precisely. However, the post-instructional structure also in- 
dicates that, for the total group, the erroneous connection of the 
binary operation-operational system pair seems to persist. Although 
interrelating operational system with the fundamental properties u.ight 
be justifiable, since ''operational system" is a somewhat ambiguous 
stimulus as an isolated phrase^ binary operation seems clearly more 
accurately considered as a defining characteristic of operational 
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systems, rather tuan as a fuudamcntal property. 

The next step in thi.o investigation was to split the students 
on the basis of their achievement scores. Within each class, the 
students were classified as above or below that class's median level 
of achievement on the pretest. The same procedure was followed for 
classification on the basis of the posttest scores. This within- 
class designation of students as "high" or '^low" was empJoyed, as 
opposed to ^ b tween-class designation based on pooling the achieve- 
m'^nt scores of the total groups, in an attempt to partially control 
for disproportionate ''class effects" attribuitable to betv/een-class 
differences such as level of general ability, aptitude, age, differences 
in teaching effectiveness, etc. Although unequal class size lessens 
the effectiveness of this technique, the class effect is at least 
composed of within-class components in the same proportion in bouii 
the high- and the low-achievement groups. 

Achievement on the pre-test was at a generally lov7 level. Out 
of 13 items, class averages hovered around 7, Visual inspection of 
the HICLUS structures resulting from the pre-test split indicated 
that the tv;o sub-groups v;ere essentially the same, i,e,, naive vis- 
a-vis operational systems,"^ 

The achievement post-test contained 29 items, and class averages 
were around 18, This higher level of achievement corroborates the 
indication, from comparing Figures and 5, that students did appear 
to benefit from the instruction. Figures 6 and 7 indicate meaning- 
ful differences between the high- and low-achievement groups on the 
posttest. According to these two structures, the major difference 
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betveen the two groups cen'cers on the manner in which the conceptual 
trio of binary operation , operational system , and ordered pair is 
interrelated with the other concepts. In Figure 6, the error per- 
sists of connecting binary operation with tl ? sub-cluster of funda- 
mental properties, whereas in Figure 7, binary operation is correctly 
connected with the concept of ordered pair , and is located within the 
sub-cluster of defining characteristics. This difference bet\;cen 
Figures 6 and 7 involves a distinction of some importance, and con- 
fusion on this matter might be expected to impair mathematical achie- 
vement • 

One way to analyze the flow of information bet^jeen the teachers 
and students of this study is to compare each teacher's graph with 
their class's graph. These graphs are presented in Figures 8 through 12 
Comparisons show that an unusual or improper placement of concepts 
in a teacher's graph frequently corresponds to a similarly improper 
conceptual organization in the class's graph. For example, in Figure 
8, teacher 10 has improperly located the identity concept with the 
cluster of concepts related to the topic of set, rather than with 
those concepts concerned with fundamental properties; this same pe- 
culiarity is reflected by his class's graph. Another example is 
found by examining Figure 9. In this figure, both teacher and class locate 
binary operation and operational system as a grouping within the 
fundamental properties cluster. Additional examples are easily located* 
indicating that just as the average teachers' structure corresponded 
closely to that of the curriculum makers, so does the ave age of a 
class's structure correspond closely to that of the teacher. 
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Additional research is planned on the topic of the present study. 
Some researcli is suggested by tne limitations and results of this wtudy 
in particular, as well as from *:i)e limitations of correlational studies 
in general. For example, even given a more refined measure of con- 
ceptual interrelations, there remains the problem of developing a 
meaningful index of the degree to which a subject's structure approaches 
a particular criterion structure. Such a measure v;ould etiable sub- 
jects to be split on the structure dimension, rather than on the achie-- 
vement dimension, as well as provide the basis for deriving a corre- 
lation between cognitive structure and achievement. Also, tighter 
experimental control, including random assignment of students to tea- 
chers and better control of the instructional process, would be highly 
desirable aspects of future investigation. Finally, the results ob- 
tained. thus far indicate that the experimental manipulation of the 
structure variable may be the most informative next step in this area. 
By mapping variables relating to structure onto a design incorporating 
experimental manipulation, many of the issues and implications of the 
present study can be more rigorously investigated. A study is cur- 
rently being planned in which experimental groups will vary according 
to whether or not they are taught a structure. Such a design can pro- 
vide data regarding causal relations between the (degree of) acqui- 
sition of a structure and student performance on achievement, reten- 
tion, and transfer tasks, hypothesized interactions involving student 
aptitudes, and the generalizability of the results of the present 
study. 



FOOTNOTE 



A ! i.::iLL. I. l;.. '.i i.!{;!.LS iL: [j /ni. I wK'd , and it 

i5ec:nis ajr^^ro.n* L... 1. .0 at. ;...L; '-oLnr. Tii-j ',r..:;u r jjc:iw..'i c ion 

does not: ijive UiV iiuiicacion o: c\Lwt,;L lo v/aicii Lite braiiLaiiV^; 

points reflect extensive or marginal agreement between subjects. 
Neither is there an indication of the alternative patterns among which 
the program had to decide^ nor of the relative strength of these al- 
ternatives. The HICLUS program might perhaps be adaptable to yield 
information on these matters. Other, perhaps non-hierarchical, clus- 
tering techniques currently being developed may prove to be of value 
for subsequent work on this problem of refining means of representing 
cognitive structure . 
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Figure 1. HICLUS representation of curriculum developers' linear graphs, 

-In each figure, these numbers indicate the order in which the groupings 
occur. For example, in this figure, i nverse and roundness constitute the 
second grouping, and the cluster of f i ni te/ i nf i ni te , opera t iona 1 system , 
e lemen t , and set constitute the seventh grouping, which is an identifiable 
sub-cluster, labeled '"sets," 
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Figure 5, HICLUS representation for all students — post-instruction 
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Figure 6, HICLUS representation for students below achievement 
medi an — post- instruction. 
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Figure 7. HICLUS representation for students above achievement 
med i an — post- instruct ion. 



ERIC 



II 



CO 
CO 




erJc 



o 



a: 

LU 
X 

o 




00 

0) 
u 



II 



to 
cn 

5 




cc 
o 



ERIC 




t)p. sys. 
Id. e 1 em • 
fund. prop, 
tommut . 



5::S^cor-%Dvo^^c>/^ assoc. 



0) 



3 



II 



CO 

to 

5 

o 




0) 
CD 



set 



00 



re 
o 
< 



ERJC 




II 



CO 

5 




Ui 
X 

o 




assoc. 



C71 



